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1 .  Linear  Programming  and  the  Simples  Method 

The  general  linear  programming  problem  may  be  stated  in  the 
following  formi 
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( 1 )  maximize 


xj  cr* 


subject  to  T _  Xj  S  t,,  i  1,2,...,  m. 

j-1  lJ  J  1 

x j  ^  j  1,2,...,  n. 

Research  hi  the  field  of  linear  programming  vras  originally  directed 
to  developing  general  motliods  for  solving  this  pre’ lorn.  The  develop- 
nonb  of  the  ni2.pl  ex  method  by  D&ntzig  (jj  provided  a  means  for  solving 
all  such  problems  in  a  relatively  efficient  and  straightforward  manner * 
The  proooss  ia  as  follows! 

1.  Inequalities  are  converted  to  equivalent  equalities  by  adding 
suitably  defined  variables.  A  so-called  tableau  arrangement  is 
then  secured  and  ar.  initial  solution  obtained  which  may  be  ex¬ 
pressed  ae 
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where  P0  represents  the  stipulations  vector  iri  th  oomponenta 
,  i»l„  on'  the  are  a  suitably  scloetod  set  of 

basis  vectors. 
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2.  Tha  elemonts  in  the  column  of  the  tableau  may  ba  regarded  aG 

coefficients  y^j  for  expressing  Pj  in  terms  of  the 

m 

(3)  )  yjj  ?! a  »  3*1 »  2,...,  n. 

3.  An  additional  row  la  located  with  elements  -  oj  whore 

m 

(4)  *j“5t  yiJ  01  *  Jil*2 . . 

4.  A  orlterlon  ie  supplied  for  judging  the  solutions  at  any  stage. 

If  all  zj  -  oj  are  non-negative an  optimum  hao  been  aohlevedi  if 

ngt,  fur-ther  improvement  is  possible. 

7 

5.  A  systematic  procedure  la  provided  for  changing  the  basis  and  con¬ 
tinuing  tha  calculations  until  on  optimum  is  achieved.  Tho  procedure 
consists  of  designating  a  vector  to  oome  Into  tho  basis  by  reference 
to  the  z  j  -  oj  and  a  vector  to  bu  removed  by  reference  to  the  ratios 
*i/yij»  yij>0* 


2.  Adjacent  Extreme  Point  Methods 

Since  the  simplex  method  was  evolved  numerous  variants  and  alter¬ 
native  procedures  ( such  as  the  dual  method  of  Lemke  (jiO  )  have  been 
devised.  In  common  with  the  simplex  method  the  prooedureo  consist  of 
movement  from  an  extreme  point  to  an  adjaoent  extreme  point  of  tho 
convex  sot  of  solutions.  The  following  propertieo  of  the  solution 
set  are  therefore  oritiool  in  all  such  procedures:  (l)  an  optimum,, 
if  it  exists,  is  attainable  at  an  extreme  point j  (2)  all  optima  can 
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be  generated  from  the  extreme  point  optima.  By  proceeding  along 
adjacent  extreme  points  it  i>  therefore  possible  to  looate  an  optimum 
if  one  exists  or  to  looate  all  optima*  if  desired,  i/ 

Extension  to  Non-Linear  Functional » 

Initially  it  was  thought  that  these  methods  (which  may  be  characterised 
as  "adjacent  extreme  point  methods")  were  applioablo  only  to  linear  function¬ 
al  s  with  constraints  given  by  linear  inequalities.  It  wss  soon  observod, 
however*  that  problems  involving  non-linear  functional s  of  the  form 


n 


which  wore  to  be  minimised  (subject  to  linear  constraints)  could  bo  reduced 
to  on  equivalent  linear  programming  problem.  ^  The  reduction  was  achieved 
by  introducing  new  variables 
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into  the  constraints  and  into  the  functional.  Every  optimum  of  the  original 
(non-linear)  problem  oould  then  be  written  as  an  optimum  of  the 

linear  problem.  Adjacent  extreme  point  methods  could  therefore  be  used  to 
locate  the  minimum  val  ues  fcvP  this  particular  class  of  convex  functional  e. 


Clearly,  then,  the  power  of  adjacent  extreme  point  methods  extended 
to  this  class  of  convex  funotionals.  It  was  thought,  however,  that  problems 
involving  minimisation  of  more  general  classes  of  convex  functionals  —  o.g.. 


1 1  See  Chomoe  D  J 
2/  See  W 


minimization  of  semi-definite  quadratic  forma  could  not  bo  solved  by 
these  methods  except  in  special  cases©  Although©  for  such  types  of 
functionals  the  optimum  is  always  on  the  boundary  of  the  solution  set© 
it  is  not  necessarily  located  at  an  extreme  point „ 

Subsequent  research  revealed  that  adjacent  extreme  point  methods  were 
capable  of  further  extensions,  Chamea  and  Leinke  [5]  noted  that  problems 
involving  minimisation  of  separable  convex  functions lo  (and  hence  maxinisa' 
tion  of  separable  concave  functionals)  could  bo  handled  by  these  methods. 
Solutions  (to  any  desired  accuracy)  could  be  achieved  by  means  of  piecewise 
linear  approximations©  Dantzig  [6]  showed  further  that  this  approach  could 
be  transformed  into  an  equivalent  formulation,,  A  different  extreme  point 
method  — -  the  bounded  variables  technique  [4]  and  [8]  •=»  could  then  also 
be  used  to  solve  such  problems© 

Still  further  extensions  were  soon  forthcoming©  In  [9]  Alan  Hoffman 
announced  that  the  method  a  of  Chames  and  Lemke  for  dealing  with  separable 
convex  functionals  had  been  extended  to  cover  the  general  problem  of  mini¬ 
mizing  an  arbitrary  convex  functional  subject  to  linear  inequalities©  He 
suggested,,  alooD  that  this  might  be  the  limit  to  which  the  simplex  method 
might  bo  pushed  in  dealing  with  problems  involving  optimization  of  non- 
linear  functionals  subject  to  linear  inequalities©  Although  full  details  of 
Dr0  Hoffman’s  work  are  not  yet  available©  his  findings  fit  rather  naturally 
into  the  evolution  of  research  directed  to  exploring  the  boundaries  of  ad¬ 
jacent  ext ’"e  me  point  methods  in  solving  linear  programming  problems© 


Objectives 

This  paper  is  intended  to  carry  those  explorations  a  stage  further. 
Specifically,  the  objectives  are  as  follows*  One,  to  show  that  it  is 
possible  to  extend  adjacent  extreme  point  methods  to  a  muoh  wider  class  of 
non-linear  functionals  --  including  functionals  which  need  not  be  either 
convex  or  concave.  Two,  to  characterise  the  olass  of  such  functionals  whied 
can  bo  embraced  by  these  methods.  Throe,  to  indicate  modifications  in  the 
simplex  procedures  and  criteria  of  optimality  which  arc  :  .joessary  to  effect 
this  extension.  Four,  to  demarcate  the  limits  of  suoh  extensions  that  moy 
bo  made  and  thus  establish  the  posrer  of  adjacent  extreme  point  Methods 
(including  the  aimplox  method)  in  dealing  with  optimisation  of  arbitrary 
functionals  subject  to  linear  constraints. 

Definition  of  Looal  Star  Optima  and  Theorem 

In  order  for  extreme  point  methods  to  be  effective  (to  the  desired 
degree  of  approximation)  it  is  clearly  necessary  for  one  of  the  following 
two  conditions  to  hold:  (l)  the  optima  are  at  extreme  points  or  (2)  the 
problem  must  be-  transformable  into  an  equivalent  one  in  which  tho  corres¬ 
ponding  optima  of  tho  original  problem  are  at  extreme*  points  of  ths  ncf! 
problem.  The  behavior  of  the  functional  at  points  othor  than  extreme 
points  la  irrelevant. 

Since  the  issue  is  optimization  in  the  largo  these  conditions,  though 
necessary,  are  not  sufficient.  Adjacent  extreme  point  methods  may,  when 
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thaae  conditions  are  fulfilled,  yield  only  looal  optima.  It  will  be  con¬ 
venient  therefore  to  define 

A  Local  Star  Optimum*  An  extreme  point 
solution  tfiich  .lolds  afunctional  value 
at  least  as  great  as  can  be  attained  at 
any  adjacent  extreme  point. 

Clearly  then  the  following  theorem  is  valid t 

Theorem*  A  noceeaary  end  sufficient 
condition  that  sdjucent  extreme  point 
methods  aivjnya  yield  .n  optimum  ia  that 
all  locul  star  optima  shall  also  be 
optima  in  the  large. 


Illustration 

The  following  illustration  will  serve  to  show  what  is  involved  and 
provide  a  basis  for  subsequent  discussion.  Suppose  that  the  objective  is 
to  minimise  the  non-linear  functional 


^  (yi#  y2»***»  yn  ) 

subject  to  a  sot  cf  linoar  constraints.  It  is  possible  to  approximate  F 
to  any  desired  degree  of  accuraoy  by  means  of  piecowiee  linear  functionals, 
provided  F  may  bo  decomposed-  by  linear  transformations  into  a  sum  of  func¬ 
tions  of  individual  variables.  Thus,  if  F  can  be  written  es 


n 

xi  -  'Ll  aiic  yk 

ksr 


where 
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th an  tho  (and  hence  F)  can  be  Approximated  (to  any  desired  degree)  by 

piecewise  linear  functional e. 

Consider,  for  example,  the  function  f(x)  sketched  in  Figure  1 


FIGURE  I 

PIECKDISE  LINEAR  APPROXIMATION 
TO  CONCAVE  AND  CONVEX  FUNCTION 

2  / 

•which  is  convex  over  certain  domains  and  oor  avo  oven-  others.—  Tho  piocowisG 
linear  approximation  is  given  by  JT*  (x).  Ti  e  k's  represent  the  slopes  of 
the  approximating  lines  and  the  a's  ropresent  the  initial  and  terminal  points 
of  each  successive  segment. 


t 

3^  Continuity  is  not  essential.. 


Numerous  convenient  devisee  are  available  for  securing  the  deairod 
degree  of  approximation,,  The  following  procedure  provides  an  example 
when  f  is  differentiable „  The  value  ~~  is  plotted  against  xc  as  in 
Figure  II»  The  area  under  gj  may  be  used  to  represent  f  (x)  and  the 
histogram  used  to  represent  f(x)„  By  choosing  appropriate 
a^»s  (i-l02(,oo.  »n)  the  absolute  difference  |  ^  may  be  made 

as  small  as  desired  and  a  suitable  approximation  thus  obtained,. 
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The  coordinates  at  extreme  points,  however,  are  coefficients  of 

V 

linearly  independent  aete  of  vectors.—  But  as  the  expressions  in  (14) 

vector  coefficient*  of 

clearly  sliow  th</  x^~  and  xjB  are  linearly  dependent.  Hence,  an  long 
as  .Tiovo;nant  is  along;  extreme  points  onlyjOither  x^j  •*  0  or  x^~  a0  bo 
that  tho  quadratic  conditions  (15)  are  automatically  satisfied  and  the 
representation  (13)  of  the  functional  la  valid.  It  ie  therefore  nscuasary 
to  stipulate  that  the  method  of  calculation  utilises  extreme  points  only. 

If  maximization  is  bolng  undertaken  and  if  the  functional  is  convex 
over  certain  ranp.os  the  ordinary  Zj  -  Cj  improvement  criteria  must  be 
modified  because  of  the  possible  appearance  of  infinite  values  for  the 
functional-^  To  fix  ideas  consider  a  problem  involving  a  non-llr.ear 
functional  which  has  bean  reduced  to  tho  form 

John* 

max.  5  c-j  Xj 

W 

(16)  subjeot  to 


( ») 
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jFT 

Bi j  ~  bi  »  1  1.2,  ...,m 

(  b) 

rt-n  ' 

^  (  i  Pin)  j  ^  1~1*2* 

T~ 

'.1=T 

(°) 

—  0 

V  fee  [3j 

h/  Seo  rule  4,  section  1.  Similar  remarks  apply  if  minimization 
i3  boing  undertaken  ar.d  the  functional  is  oonoavo  *sr  certain 
range  s . 
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where  the  restrictions  (a)  are  associated  with  the  original  problem  and 
the  restrictions  (b)  are  adjoined  (as  in  the  Illustration  of  the  preced¬ 
ing  section)  In  the  process  of  securing  the  approximation.  Note:  (l) 
that  the  object  in  the  approximation  is  to  obtain  a  value  which  agrees 
with  fj  (xj)  on  the  oxtrene  points,.  (2)  that  different  numbers  of 
subdivisions  may  be  used  for  each  fj  (xj)^and  (3)  so  long  as  movement  is 
restricted  to  extreme  points  the  quadratic  condition  ie  automatically 
satisfied  and  may  thus  be  regarded  as  redundant. 


In  more  compact  vector  notation,  (16)  iriay  be  reexpressod  an 


(17) 


&fP 

*“•  II  °J  Xj 
J=i 

subject  to 
n 


£  (Sj)  Xj  +  x*  =(»:) 

X[  -0 


are 


where  the  P's,  D's  and  G's,  (as  well  as  0)^ appropriately  defined  oolumn 
vectors  of  m  and  n'  rows  respectively. 


Evidently  any  linearly  independent  set  of  a  Pj'e  must  be  associated 
with  a  sot  of  a'  linearly  independent  G j 8 s  if  movement  is  to  be  restricted 
to  extreme  points.  Hence  rule  5  of  the  simplex  routine  which  stipulates 
replacement  of  a  Fj  by  a  to  provide  a  succeeding  basis  must  be  modified 
ao  that  a  can  only  be  replaced  by  a  ( 1  ^  \  and f?  \  by  a 

V  Vbl  vtj 

Hint  optiisility  airltarion  imijt  then  ’*»  :yilTio:t  ton  '<j  -  n  >  0  a1?,  per- 

mles-Luio  .-uptuC'nn  sits , 


tfcr  applying  these  ruleo  to  the  extended  Mt,  aovmoot  Its  restricted 
to  aoctr— »  points  of  the  original  Mt.  Home*  an  optima  v/hich  is  attained 
for  the  enlarged  pro  him  of  employing  these  altered  procedures  and  criteria 
Is  also  an  options  for  the  original  pro  him.  In  general,  only  a  local  star 
optimum  is  guaranteed,,  When  a  local  star  optimum  is  also  an  optimum  in  the 
large  the  procedures  and  criteria  outlined  a  bore  trill  suffice  to  locate  this 
point  irrespective  of  the  non-linear  characteristics  of  the  functional,, 

BiaagE  saiBBiffltt 

The  procedure  is  capahlo  of  further  extension  in  particular  applica¬ 
tions,,  Clearly  all  cases  in  ubich  local  star  optima  ore  optima  in  the  large 
are  oonprehended  by  the  above  rules  and  criteria.  In  other  cases  it  is 
possible  to  employ  these  net.'  criteria  to  reach  various  local  star  optima 
by  starting  at  uidely  separated  extreme  points.  It  may  than  be  possible 
to  establish  that  an  optimum  optlmorum  baa  been  attained  in  the  process. 

The  ability  to  use  adjacent  extreme  point  methods  (such  as  the 
elmplex  procedure)  as  a  general  procedure  for  solving  all  such  problems 
is  patently  dependent  on  tho  development  of  systematic  and  efficient  means 
for  traveredug  local  star  optima 0  At  present  no  such  general  techniques 
are  available.  The  procedure  outlined  above  is,  however,  amenable  to 
heuristic  employment  when  a  priori  considerations  do  not  make  it  apparent 
that  a  local  star  optima  is  also  an  optima  in  the  large.  On  attaining 
a  local  star  extremum  it  nay  be  possible  to  establish  directly  that  it  is 
also  an  optimum  optimum.  The  current  lack  of  general  and  efficient  trav¬ 
ersal  techniques  demarcates,  for  the  present,  the  tan-linear  power  of  adjacent 


ext: 


point  methods  of  linear  programing, 
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